On Monotonicity of the Optimal Transmission Policy in Cross-layer Adaptive m-QAM Modulation

This paper considers a cross-layer adaptive modulation system that is modeled as a Markov decision process (MDP). We study how to utilize the monotonicity of the optimal transmission policy to relieve the computational complexity of dynamic programming (DP). In this system, a scheduler controls the bit rate of the m-quadrature amplitude modulation (m-QAM) in order to minimize the long-term losses incurred by the queue overflow in the data link layer and the transmission power consumption in the physical layer. The work is done in two steps. Firstly, we observe the L-convexity and submodularity of DP to prove that the optimal policy is always nondecreasing in queue occupancy/state and derive the sufficient condition for it to be nondecreasing in both queue and channel states. We also show that, due to the L-convexity of DP, the variation of the optimal policy in queue state is restricted by a bounded marginal effect: The increment of the optimal policy between adjacent queue states is no greater than one. Secondly, we use the monotonicity results to present two low complexity algorithms: monotonic policy iteration (MPI) based on L-convexity and discrete simultaneous perturbation stochastic approximation (DSPSA). We run experiments to show that the time complexity of MPI based on L-convexity is much lower than that of DP and the conventional MPI that is based on submodularity and DSPSA is able to adaptively track the optimal policy when the system parameters change. Index Terms *Ni Ding is with the Research School of Engineering, College of Engineering and Computer Science, the Australian National University (ANU), Canberra, ACT 2601, Australia (email: {ni.ding}@anu.edu.au). Parastoo Sadeghi and Rodney A. Kennedy are with the Research School of Engineering, College of Engineering and Computer Science, the Australian National University (ANU), Canberra, ACT 2601, Australia (email: {parastoo.sadeghi, rodney.kennedy}@anu.edu.au). September 7, 2015 DRAFT IEEE TRANSACTIONS ON COMMUNICATIONS 2 f (t) scheduler FIFO queue m-QAM transmitter wireless fading channel data link layer physical layer Fig. 1. Cross-layer adaptive m-QAM system. f (t) denotes the number of packets arrived at data link layer at time t. The packet arrival process {f } is random. The scheduler controls the number of bits in the QAM symbol in order to minimize the queue overflow and transmission power consumption simultaneously and in the long run. cross-layer adaptive modulation, dynamic programming, L-convexity, Markov decision process, stochastic approximation, submodularity.